Supporting Parallel Applications on Clusters of Workstations: The Intelligent Network Interface Approach

This paper presents a novel networking architecture designed for communication intensive parallel applications running on clusters of workstations (COWs) connected by high speed networks. This architecture permits (1) the transfer of selected communication-related functionality from the host machine to the network interface coprocessor, and (2) the exposure of this functionality directly to applications as instructions of a Virtual Communication Machine (VCM) implemented by the coprocessor. The user-level code interacts directly with the network coprocessor as the host kernel only ’connects’ the application to the VCM and does not participate in the data transfers. The distinctive feature of our design is its flexibility: the integration of the network with the applicationcan be varied to maximize performance. The resulting communication architecture is characterized by a very low overhead on the host processor, by latency and bandwidth close to the hardware limits, and by an application interface which enables zero-copy messaging and eases the port of some shared-memory parallel applications to COWs. The architecture admits low cost implementations based only on off-the-shelf hardware components. Additionally, its current ATM-based implementation can be used to communicate with any ATM-enabled host. We use three applications to demonstrate the high performance of the architecture’s current implementation: (1) a synthetic client/server application, (2) a parallel implementation of the Traveling Salesman Problem, and (3) a parallel engine for discrete event simulation(GTW). The distributedand shared-memory versions of these applications have comparable performance. Furthermore, the VCM-based approach to creating a clustered, parallel machine is shown to scale well in terms of matching required with offered communication bandwidths. Partially funded by DARPA grant DABT63-95-C-0125 and by NSF Grants CDA-9501637, CDA-9422033, ECS-9411846, and MIP-94085550